Dynamic control of surgical instruments in a surgical robotic system

ABSTRACT

A robotic surgical system configured to control movement of a first instrument and a second instrument, each of which is on a robotic manipulator. In described modes of operation, movement of the first instrument is surgeon controlled based on surgeon input to the robotic system. Movement of the second instrument is also surgeon controlled, but its motion is defined by the chosen mode of operation which sets the amplitude and direction of the second instrument&#39;s motion relative to the actual or instructed motion of the first instrument. In this way, two instruments are simultaneously moved based on input from a single surgeon input device.

BACKGROUND

There are various types of surgical robotic systems on the market orunder development. Some surgical robotic systems use a plurality ofrobotic arms. Each arm carries a surgical instrument, or the camera usedto capture images from within the body for display on a monitor. SeeU.S. Pat. No. 9,358,682, which is incorporated herein by reference.Other surgical robotic systems use a single arm that carries a pluralityof instruments and a camera that extend into the body via a singleincision. See WO 2016/057989, which is incorporated herein by reference.Each of these types of robotic systems uses motors to position and/ororient the camera and instruments and to, where applicable, actuate theinstruments. Typical configurations allow two or three instruments andthe camera to be supported and manipulated by the system. Input to thesystem is generated based on input from a surgeon positioned at a masterconsole, typically using input devices such as input handles and a footpedal. Motion and actuation of the surgical instruments and the camerais controlled based on the user input. The image captured by the camerais shown on a display at the surgeon console. The console may be locatedpatient-side, within the sterile field, or outside of the sterile field.

US Patent Publication US 2010/0094312 (the '312 application),incorporated herein by reference, describes a surgical robotic system inwhich sensors are used to determine the forces that are being applied tothe patient by the robotic surgical tools during use. This applicationdescribes the use of a 6 DOF force/torque sensor attached to a surgicalrobotic manipulator as a method for determining the haptic informationneeded to provide force feedback to the surgeon at the user interface.It describes a method of force estimation and a minimally invasivemedical system, in particular a laparoscopic system, adapted to performthis method. As described, a robotic manipulator has an effector unitequipped with a six degrees-of-freedom (6-DOF or 6-axes) force/torquesensor. The effector unit is configured for holding a minimally invasiveinstrument mounted thereto. In normal use, a first end of the instrumentis mounted to the effector unit of the robotic arm and the opposite,second end of the instrument (e.g. the instrument tip) is located beyondan external fulcrum (pivot point kinematic constraint) that limits theinstrument in motion. In general, the fulcrum is located within anaccess port (e.g. the trocar) installed at an incision in the body of apatient, e.g. in the abdominal wall. A position of the instrumentrelative to the fulcrum is determined. This step includes continuouslyupdating the insertion depth of the instrument or the distance betweenthe (reference frame of the) sensor and the fulcrum. Using the 6 DOFforce/torque sensor, a force and a torque exerted onto the effector unitby the first end of the instrument are measured. Using the principle ofsuperposition, an estimate of a force exerted onto the second end of theinstrument based on the determined position is calculated. The forcesare communicated to the surgeon in the form of tactile haptic feedbackat the hand controllers of the surgeon console.

In a laparoscopic surgical procedure performed using manual instruments,the surgeon manipulates the primary instruments while a surgicalassistant controls the camera and third instrument. Giving the surgeoncontrol over the camera and the third instrument is an importantimprovement over traditional laparoscopy, these two implements are nolonger controlled dynamically as they are when manually handled by auser.

U.S. Pat. No. 9,360,934 describes a robotic system allowing dynamicsurgeon control of the robotic manipulator that supports the camera byallowing the surgeon to control the camera using an eye tracking system.The other two or three robotic manipulators carrying surgicalinstruments are driven via handles in a surgeon console. Since thesurgeon has just two hands, operation of the system in proceduresutilizing more than two surgical instruments on robotic manipulatorsrequires the surgeon to choose which two instruments s/he will controlusing the console at any given moment. This application describescertain modes of operation that enable dynamic, surgeon-controlledmovement of a third instrument while also controlling the two primaryinstruments.

Although the inventions described herein may be used on a variety ofrobotic surgical systems, the embodiments will be described withreference to the system shown in FIG. 1. In the illustrated system, asurgeon console 12 has two input devices such as handles 17, 18 that thesurgeon selectively assigns to robotic arms 14, 15, 16, allowing surgeoncontrol of two of the surgical instruments 10 a, 10 b, and 10 c disposedat the working site at any given time. To control a third instrumentdisposed at the working site, one of the two handles 17, 18 isoperatively disengaged from one of the initial two instruments and thenoperatively paired with the third instrument. (Note that in FIG. 1 thelaparoscopic camera, which may be a robotically positioned camerasupported by a fourth robotic arm, is not shown for purposes ofclarity.)

A control unit 30 is operationally connected to the robotic arms and tothe user interface. The control unit receives user input from the inputdevices corresponding to the desired movement of the surgicalinstruments, and the robotic arms 14, 15 and 16 are caused to manipulatethe surgical instruments accordingly.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates elements of a surgical robotic systemof a type that may be adapted for use with the disclosed invention.

FIGS. 2 and 3 schematically illustrate the use of robotically-controlledtraction and surgeon-controlled traction according to an aspect of thedisclosed invention.

DETAILED DESCRIPTION

The present application describes a system and method for maintainingoptimal tissue tension between instruments of a surgical robotic system.

The surgical system may be of a type described in the Background, or anyother type of robotic system used to maneuver surgical instruments at anoperative site within the body. In some embodiments, the surgical systemis one that includes sensors positioned to estimate the forces eachrobotically manipulated surgical instrument applies to tissue.Configurations such as those described in US Patent Publication US2010/0094312, or other configurations, can be used for this purpose.

In this description, the terms “primary instruments” will refer to thesurgical instruments moveable by robotic manipulators based directly oninput delivered to the system by the surgeon inputs at the surgeonconsole. The term “secondary instrument” or “third instrument” willrefer to a third instrument that is also moveable by a roboticmanipulator, but whose movement characteristics (e.g. direction,orientation) is based in some way on the characteristics of the directedmovement of the one of the primary instruments that is operating underthe control input from the surgeon console. While it will be typical fortwo primary instruments to be used, in some embodiments there might justbe one primary instrument. For example, there might just be one primaryinstrument in a configuration where only one instrument is beingcontrolled by a user input handle because the other user input handle iscontrolling the camera rather than using eye tracking or some other formof input.

This application describes two modes that may be implemented to controlmovement of a secondary instrument. These are referred to as the modesof Mirrored Motion and Matched Motion. When entering one of these modesof operation, control of the secondary instrument is such that thesecondary instrument either follows the movement of one of the primaryinstruments, or mirrors the movement of one of the primary instruments.

The surgeon console includes a user interface that allows the user toselect a mode of operation. The user interface used to select the modemay include any type of input device, such as a button or other input onthe input handles, a graphical user interface, voice input, eye trackinginput, foot pedal, keyboard, touch sensitive surface, etc. Because thesemodes of motion may only be desired for specific tasks, it is preferredthat the user interface allow these modes to be easily activated, andde-activated, when desired. The user interface may also allow the userto select the instrument that is to function as the primary instrumentand the instrument that is to function as the secondary instrument.

For example, using the example of buttons on each of the surgeon inputhandles, if the button held is on the left handle of the user interface,the left instrument would be the primary instrument and the secondaryinstrument would either follow in matched or mirrored motion. Whereas ifthe button is held on the right handle, the right instrument would bethe primary.

Turning first to mirror motion, this is a mode in which motion of thesecondary instrument mirrors the motion of a primary instrument. Thedetermination of the direction of motion of the secondary instrumentmight be determined based on the surgeon input at the console handle forthe primary instrument, or it might be determined in another way (e.g.by directly monitoring the position of the primary instrument usingcomputer vision or by calculating its position using sensors associatedwith the robotic manipulator supporting it).

Use of mirror mode involves first defining the plane between the primaryinstrument and the secondary instrument about which the motion ismirrored. FIG. 2 illustrates a primary instrument and a secondaryinstrument separated by a plane. When the mirrored mode of operation isengaged, the tip of the secondary instrument will follow the firstinstrument tip as if mirrored by the plane. If the first instrumentmoves up, down, left, or right, relative to the mirror, the secondinstrument will follow in amplitude and direction. However, if the firstinstrument moves towards the mirror plane, the second instrument willalso approach the mirror plane. Likewise, if the first instrument movesaway from the plane, the second instrument will move away from the planeas well.

It should be noted, that, unlike a typical mirror, the starting positionof the instrument tips relative to the mirror plane may not beidentical. As an example, the first instrument may intersect the mirrorplane at a different location than the second instrument when they bothapproach the mirror plane. Typically, though, the amplitude of themovement of the primary and secondary instruments, from their respectiveorigins, is the same and the direction of the motion, relative to themirror plane is maintained.

The control of the jaw opening and closing may be optionally included inmirrored motion operation. In some embodiments, the surgeon mayselectively turn on/off mirrored jaw motion when within the mirror mode.

The plane about which motion is mirrored could have a default positionand orientation programmed into the system. For example, the plane couldbisect the angle between the instruments, or the plane could beperpendicular to the camera view. Alternatively, the user may be able toselect the desired plane using the user interface, and optionally toadjust the plane during use of mirror mode. Alternatively, theorientation may be selected or adjusted by the system. For example, asystem equipped with capabilities to assess tissue planes (for exampleusing structured light techniques) and/or cavity volumes (time offlight, 3D, etc.) may be able to adjust the orientation of the mirrorplane to optimize the movement of both the primary and secondaryinstruments while avoiding inadvertent contact between either andadjacent tissue. For example, in tight working spaces, where there isless room for lateral, side-to-side, instrument motion, the mirror planemay be adjusted such that the instruments are either pulled or pushedalong an insertion axis of the instrument.

In the “matched” mode of operation, the first and second instrumentsmove in tandem, based on the amplitude and direction of the motion ofthe first instrument. See FIG. 3. This mode differs from mirrored modeprimarily in that matched motion does not have any degree of freedomthat is reflected, as in a mirror. The amplitude and direction of thefirst instrument is replicated by the second instrument, from its pointof origin. In a variation of this embodiment, a user-selected orsystem-selected scaling factor may be applied, so that the amplitude ofmovement of the secondary instrument is lower or higher than that of theprimary instrument by some scaling factor. In another variation, themovement of the instruments may be matched in amplitude, but thedirection may be differ. In this embodiment, the direction of movementmay be defined by the vector of the instrument shaft (e.g. along thelongitudinal axis of the shaft). The vector for each instrument may bedetermined using the kinematics of the robotic manipulator, and/or basedon the interoperative view captured by the laparoscope (using, forexample, a computer vision algorithm to detect the instrument and itsorientation). Matched mode may be used for a variety of applications,including one that allows the secondary instrument to be kept inproximity to the primary instrument. This allows the secondaryinstrument to remain in the surgeon's field of view on the visualdisplay, so the surgeon can readily exit matched mode and then assignthe instrument that had been serving as a secondary instrument to ahandle of the console, allowing direct control of that instrument withthe handle.

The described modes allow for dynamic control over a third instrument ina robotic surgical system where first and second instruments arecontrolled using surgeon inputs, thus letting the surgeon operate threeinstruments with two hands in specific scenarios by configuring thesystem so that two surgical instruments (a primary and a secondary) maybe driven based on the movement of a single master input device.

All patents and applications referred to herein, including for purposesof priority, are incorporated herein by reference.

We claim:
 1. A surgical method, comprising: providing a surgical roboticsystem having a first robotic manipulator with a first surgicalinstrument thereon, a second robotic manipulator with a second surgicalinstrument thereon, and first and second user input devices; receivinguser input from the first input device and the second input device;receiving user input to operate the surgical robotic system in a firstmode of operation; in the first mode of operation, simultaneously movingeach of the first and second surgical instruments in response to userinput from only the first user input device, wherein said moving stepincludes causing movement of the second surgical instrument that mirrorsmovement of the first surgical instrument relative to a mirror planebetween the first instrument and the second instrument; and receivinguser input to operate the surgical robotic system in a second mode ofoperation; in the second mode of operation, moving the first surgicalinstrument in response to user input from the first user input device,and moving the second surgical instrument in response to user input fromthe second user input device.
 2. The method of claim 1, whereinsimultaneously moving each of the first and second surgical instrumentsincludes moving the first surgical instrument with a first amplitude andfirst direction and moving the second surgical instrument with a secondamplitude and second direction, wherein the second amplitude is equal tothe first amplitude, and wherein the second direction is mirror oppositeto the first direction.
 3. The method of claim 1, wherein the providingstep provides the first surgical instrument to have first jaws andfurther provides the second surgical instrument to have second jaws, andwherein the method includes: in both the first mode of operation and thesecond mode of operation, closing the first jaws in response to inputfrom the first input device; and in both the first mode of operation andthe second mode of operation, closing the second jaws in response toinput from the second input device.
 4. The method of claim 1, whereinthe first surgical instrument has a first longitudinal axis and thesecond surgical instrument has a second longitudinal axis, and whereinthe mirror plane bisects an angle formed between the first longitudinalaxis and the second longitudinal axis.